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ABSTRACT 

The research reported in this report is, considering the hypotheses, a replication of an earlier investigation with a different 
experimental population and in a different experimental setting. Where, in the earlier study, regular university students were 
involved, in the present study we focua on students studying in a distance education and open learning setting. One of the 
methodological difficulties with this research population is to reach a large and representative sample. As a consequence, the 
experimental group in the present study is limited in number, thus preventing to obtain statistically significant results in relation 
to all hypotheses tested. 

Notwithstanding these methodological difficulties, the results are helpful to draw conclusions in relation to the quality and 
impact of expertise. As in the earlier study, attention was paid to the construction of a set of tests to grasp the complex nature 
of prior knowledge at the content level. The instruments used during the earlier study were improved in relation to their 
psychometric qualities. 

Concerning the hypotheses about the quality of expertise, the results of this investigation revealed that students with different 
educational backgrounds and studying different diploma-types, possess a complex of different expertise components. 
In relation to the impact of prior knowledge on the acquisition of subject-oriented knowledge, regression analysis reveals that 
expertise accounts for 37-42% of the variance in posttest scores. The optimization of the instruments used was helpful to 
differentiate to a better extent the impact of the three different expertise tests. 

Since the results of the present study are consistent with the findings of the earlier study in relation to the quality and impact of 
expertise, more weight can be given to the arguments that can be put forward to support students, in the initial stage of their 
studies, to cope with the demands of specific domains to be studied. 
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L INTRODUCTION. 

Earlier research and overviews of recent theories and research concerning the effect of prior knowledge 
(expertise) on learning, indicate that prior knowledge is one of the most important educational variables 
(Dochy, 1988). We define prior knowledge or expertise as the domain-specific knowledge and skills, 
available before endeavouring a specific learning task. A domain is considered to be the total knowledge 
base related to an academic discipline, for example psychology, medical science or economics. In our case, 
this domain is 'economics' which can be sub-divided into different subject matters, e.g. accountancy, 
financing, etc. 

In order to investigate prior knowledge phenomena and their potential impact on the learning process, we 
conducted a series of experiments with students enroled for the course 'Economics and Money'. This is a 
multi-functional course, which means that it is part of and supposed to fit for all students in different 
curricula within the university. 

This study is a "replication" study, considering it's hypotheses. Among the different reasons for setting up 
this replication approach to the research questions about the quality and impact of expertise, the particular- 
ities of the Ou population is of primary importance. 

Earlier experiences showed that it is extremely difficult to involve a large number of Open university 
students (heterogeneously, decentralization) in such a study. Because the Maastricht student population, for 
reasons like attainability and continuation, is much easier to investigate, former research has been carried 
out with Maastricht university students. From this investigation we concluded that contextual and personal 
variables are not valid indicators concerning a students' prior knowledge (Dochy, e.a., 1991a). This is in 
accordance with the findings of earlier research (Powell, Conway & Ross, 1990). Moreover, this already 
suggest that the Maastricht research results - although collected from a population with different contextual 
and personal characteristics - might be transferred to the Ou-research and education context. However, more 
research is needed to support the possibility and acceptability of this transfer. This can be called our 
"context-transfer approach". The main hypotheses of the present research have already been tested with the 
regular university student population (Dochy, et.al, 1991b). To look for additional support for the results 
obtained and to look for support for our context-transfer approach, a replication study has been set up. 
The specificity of this approach is described in part 2 of the text when discussing the theoretical back- 
ground. In our approach, special attention is paid to the construction of a set of expertise tests to measure a 
complex of expertise components. After summarizing the research procedure and the research results, 
implications of the present study for future investigations are presented. 
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2± THEORETICAL BACKGROUND. 

Both from an experiential point of view ( Miller Cleary, 1989) and from the researcher's point of view 
(Glaser, 1984) the importance of prior knowledge is stressed. Glaser (1984) states for instance that in. 
education and thinking, "people continually try to understand and think about the new in terms of what they 
already know". Earlier research and our own research have tried to detect the impact and the quality of 
expertise. We review - in short - some basic research results in order to put the present research in context. 

Although - in literature - the impact of prior knowledge is often stressed to this extent that all learning might 
even depend on it (Resnick, 1983), also other factors like student characteristics do influence the learning 
process and can interact with the impact of prior knowledge. But, it is yet not clear which personal or 
contextual variables play a significant role in this context (Ferguson-Hessler, 1989). Moreover, research 
indicates that, if different variables are taken into account, "prior knowledge" always has the strongest 
general effect and overrules other variables in descriptive and explanatory models (Ethington, 1990; 
Bruinsma and Geurts, 1988). This pre-dominance of -jxpertise in learning brings Glaser (1987) to the 
conclusion that the assessment of prior knowledge should be stressed or studies should be conducted to 
assess the knowledge state of the learner. A common practice in this perspective is the explication of the 
rules that can account for systematic error patterns in task-performance. The explication of these rules can 
help to construct explanatory models of understanding. Typical examples of this approach are Siegler's rule 
assessment approach in science concepts (1981) and Brown and Burton's "buggy" system in arithmetic 
computation (1978). 

In our own research, expertise was analyzed in relation to an introductory course in economics. 
The results of a first project suggested that differences in expertise are to be found between economics 
students (ES) and law students (LS). These differences were derived from differences in pass rates and 
number of examination trials (Dochy & Bouwens, 1990). 

A further investigation of 'the Heerlen Group' revealed that these differences are not significant in terms of 
differences in university test scores. Nevertheless, multiple classification analysis (MCA) 1 shows very 
consistent trends : economics students score systematically higher than law students. It could also be sho^vti 
that no personal or contextual variables are useful as indicators of a student's prior knowledge (Docl v , 
Bouwens, Niestadt, Wagemans, 1991). In the latter study, 76 variables (i.e age, sex, prior education, 
motives, work experience, etc.), which could give a direct or indirect indication of a subject's expertise, 
were introduced. 

The research results discussed above have in common that they stress the importance of future research, 
focusing on the detection of specific measures of expertise which can shed light on specific components of 
expertise. The present research is an attempt in this direction : we want to grasp a students' prior 
knowledge and its impact on learning, by concentrating the initial focus on the construction of a set of prior 
knowledge state tests. In past research, existing course-related tests were mostly used to assess prior 
knowledge (De Corte, 1990), without differentiating between types of expertise along certain dimensions 
(Dochy, 1990). 

The current research invoked different tests, varying along the "content dimension". In later investigations, 
we will search the influence of educational, epistemological and psychometrical dimensions. 
In order to define the variants along the content dimension, content experts (i.e. economists) were asked to 
ascertain types of content knowledge that influence learning results. Moreover, we also reviewed the types 
of tests used in literature. 

Within the tests, used to grasp an individuals' mastery of the domain, we can distinguish a tripartite, i.e. 
subject-oriented knowledge state tests (SO-KST), mathematics tests (MA-KST) and optimal requisite 
knowledge state tests (OR-KST). 



A multiple clauification analysts (MCA) shows how the icorei of the specific tub-groups ire relited to the mean (grand mean) of 
the total population. In a MCA-uble the mean deviation of the tub-group Korea to the grand mean it shown. If the analysis of 
variance takes into account the influence of a co-variable (analysis of co- variance), the adjusted deviation for independent variables is 
used in our MCA-tables. 

o 
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Figure 1 gives an overview of these types of different tests in relation to the domain of economics. 



Domain Economics 



Micro-economics Accountancy Financing x 




Domain 
n 



Mathematics Knowledge State Test 



Optimal Requisite Knowledge State Test 



SO KST: 
SO KST 2: 



OR KST: 
MA KST: 



Subject-oriented knowledge state test 
Posttest 



Optimal requisite knowledge state test 
Mathematics knowledge state test 



| Subject -oriented tests 
Cross-section tests 



Figure 1 : Different knowledge stale tests in relation to the domain. 

This investigation focuses first of all on subject-oriented knowledge (SO) and a cross-section of SO 
knowledge, i.e. optimal requisite (OR) knowledge and mathematics (MA) knowledge. Optimal requisite 
knowledge is this part of prior knowledge that is, according to content experts, necessary to start the study 
of the learning task/course under optimal conditions. As shown in figure 1, the MA KST and the OR KST 
do contain items that do exceed above the subject-oriented knowledge of micro-economics. In this sense 
they are cross-domain tests. Our earlier research showed that these tests are promising in predicting student 
performance on a learning task. The predictive power was mainly related with mathematics and optimal 
requisite knowledge (Dochy, Valcke and Wagemans, 1991). Based on this and other earlier research 
(Dochy, 1988), we expect that higher scores on subject-oriented and cross-section expertise tests will result 
in higher scores on a posttest. 

The Dochy, Valcke and Wagemans study (1991) also revealed that there are striking differences in 
mathematics expertise and optimal requisite knowledge between student types (economics students (ES) and 
law students (LS)). 

An additional approach to grasp the prior knowledge level of the students, was to ask the students 
themselves to estimate their mastery level. In an experiment by Lodewijks (1981), it was shown that prior 
knowledge could account for 36 % of the explained variance in posttest scores. Prior knowledge was 
operationalized here as the subjective rating of the familiarity with the content of the learning task. 
However, when considering the Falchikov and Boud (1989) meta-analysis of 48 student self-assessment 
studies, the researchers came to more differentiated conclusions: most studies found positive effect size.>, 
indicating overrating on the part of the students, more experienced students in a particular field (or with 
more expertise) are more accurate estimators, and students taking introductory courses appear to self-assess 
particularly inaccurate. 
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3, RESEARCH DESIGN. 

3.1 Hypotheses 

Taking into account the theoretical background of the current investigation, the following three groups of 
hypotheses can be put forward 2 : 

1 . The quality of expertise in ES and LS : 

- The overall expertise of ES and LS does not differ. 

- ES and LS do possess a different composition of expertise components. 

2. The impact of expertise and expertise levels on knowledge acquisition : expertise influences the 
posttest scores. 

- Specific expertise components influence the posttest scores. 

- Students with better expertise obtain higher scores for the posttest scores. 

3. Student estimation of their expertise level does not reflect their real expertise level. 

3.2 Research population 

In September 1989 an experiment was set up in which students of the Maastricht university participated in 
order to investigate the role of expertise in studying a learning task (Dochy, Valcke and Wagemans, 1991). 
In the present study, students learning in a distance education setting, in particular students at the Open 
university were involved (januari 1991). 

It is difficult to reach a large and representative sample from this research population. Our earlier research 
showed several reasons for this. First, one has to take account of the regular drop-out of nearly 60 % at 
distance teaching universities. Secondly, those students choose the open learning context deliberately 
because of the large degrees of freedom: free choice of study time, study place, study pace, etc. An 
experiment at a certain time, at a certain place, in which has to be studied at a certain pace is thus not the 
first choice of such students. Thirdly, also resulting from the open learning context is the conclusion that for 
simple questionnaires, 50 to 60 % of the respondents answer, but for true learning tasks in experimental 
settings only 10 % of the subjects persist. As a consequence, the experimental group in the present study is 
limited in number (n=15), thus preventing to obtain statistically significant results in relation to all 
hypotheses tested. In order to reach these students, we took a representative sample of students newly 
enroled for the course 'Economics and Money'. The students of this sample, all registered at the Ou study 
centre in Amsterdam, were asked if they were able to participate in the experiment. 

From the final group of respondents who participated in the experiment, 7 students were identified as ES, 4 
students as LS, and the rest studied a combination of disciplines or had not decided on a specific discipline 
yet. 

3.3 Instruments 

3.3.1 Description of the research instruments 

In this investigation the construction of a set of four short tests was based on the following considerations: 

in order to detect the differential role of components of prior knowledge, the tests should reflect these 
components; 

the student load for completing the tests should be restricted; 

the research procedure should be restricted to the time limit of one day; 

within this time limit there should be enough time for students to complete the learning task. 



Although a third category of ttudenu (other studenu»OS) were involved, we will mainly focus our hypotheses and analyses on ES 
andLS. 
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The following tests (1.1 versions) are used : 

A SO-KST1: This test includes 34 multiple-choice items (4 alternatives) and is related to learning 
units 14 and IS of the "Economics and Money" course. Validity of the test was checked by content 
experts who evaluated if the items were representative for the subject-matter. 

An OR-KST: This test consists of 8 items (open-ended or multiple-choice), each of them representing 
a set of sub-items (17 in total). Construction of this test was based on the opinion of economics 
experts who identified and explicitated the optimal requisite knowledge for the execution of the 
learning task. 

A MA-KST: This test, cross cutting the domain, contains 28 items. The test is based on a self- 
evaluation test of the Antwerp university. 

A posttest (SO-KST2): This test, consisting of 34 items is a parallel test-version of SO-KST1. 
A questionnaire about background variables and experiential learning. This questionnaire was handed 
out at the end of the experiential session with the request to fill in at home and to return within a 
week to the Open university. 

3.3.2 Psychometric qualities of the research instruments 

To assess the reliability of the tests, the a-coefficient was calculated based on a first version of the tests 
(1.0), as used in a former experiment (Dochy, Valcke and Wagemans, 1991). 

Table 1 : Reliability of 1 .0 tests. 



alpha 


SO-KST 


.AO 


OR-KST 


.68 


HA-KST 


.82 


S0-KST2 


.23 



Item-test correlation for all 1.0 versions of the tests revealed that there are no items with negative or low 
correlat >ns to the total score. Nevertheless, reliability is to be considered as rather low, mainly due to the 
limited amount of items in the tests. It is generally accepted that a test needs 40 items to reach a reliability 
of .80. For the MA-KST holds that the original Antwerp test (Dyck, s.d.) had a reliability of .85. 

In order to bring the reliability of all 1.0 versions of the tests up to .80 or more, we used the general 
Spearman-Brown prophecy formula. To estimate what the reliability of a test would be if it were made 
longer the prophecy equation is 

K . r 



1 + (K - 1) r 



where 

r„ = predicted reliability of a test K times as long as test version 1.0 
r = reliability of test version 1.0 

K = ratio of number of items in test version 1. 1 to number of items in test version 1.0 

According to our calculation, all 1.1 versions are approximately three times longer than the 1.0 versions 
(see table 2, after correction), with the exception for the MA-KST which had already a sufficient reliability. 

During test-construction special attention was paid to content validity. This was realized by involving 
content experts (SO-KST 1 & 2, OR-KST) or by using valid item banks (MA-KST). 
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To assess the reliability of the 1.1 versions of the tests (after enlargement based on the Spearman - Brown) 
formula, the cr-coefficient was calculated. Item-test correlation for all tests revealed negative or low 
correlations for some items. After omission of these items, the alpha coefficient was calculated for the 
corrected test versions (see table 2, after correction). 

Table 2 : alpha-coefficients of the different test-versions 1.1 





Without 
correction 


After 
correction 


SO KST 


.51 


.63 


OR KST 


.94 


.96 


HA KST 


.92 


.93 


POSTT. 


.84 


.85 



3.4 Research procedure 

The research procedure consists of 10 phases : 

- Registration and introductory session. 

During this session, the main aim of the research project and the research procedure was outlined. 

- The administration of the four expertise tests : 
SO-KST1 (± 50 min.) 

OR-KST (± 40 min.) 
MA-KST (± 45 min.) 

- Lunch time (30 min.) 

- A study period (±90 min.) : during this study period the students studied the text of learning unit 14 
& 15 or the course "Economics & Money". The study task was limited to the individual going through 
the course text. There was no control of individual approaches towards the study task. Text-support 
was equal to the regular support provided in Ou-courses. 

- Administration of the posttest SO-KST2 (± 50 min.). 



The administration of a set of expertise tests was a specific feature of this research as explained in the 
introductory parts of this text. 

The overall procedure was timed according to a strict time schedule, while avoiding time stress or fatigue. 
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4,. DISCUSSION OF THE RESEARCH RESULTS. 
4.1 General results 

Table 3 gives an overview of the mean and the standard deviation of the student-scores for the four different 
prior knowledge tests and the posttest : 

Table 3 : mean acoret for the prior knowledge test* 
and the poatteat 





m 


a 


SO KST 


10.27 


3.89 


OR KST 


9.00 


10.53 


MA KST 


8.60 


7.34 


POSTT. 


13.20 


6.23 


PKST1 


27.87 


19.45 


PKST2 


19.26 


12.78 



To calculate a general measure of expertise, the scores for the three prior knowledge tests have been added 
to each other (PKST1). Correlation analysis between the four tests reveals that the optimal requisite test and 
the mathematics-test do correlate to a very high extent (.891~) 3 . This is to be expected, since the optimal 
requisite test contains items, based on mathematics. Since both tests measure - to a certain extent - the same 
type of expertise, a second general measure of prior knowledge has been calculated, excluding the scores 
for the mathematics test (PKST2). 

The mean and standard deviation of both general measures of expertise can also be found in table 3. 



4.2 The quality of expertise in studying economics. 

4.2.1 The overall expertise of students of different diploma type does not differ. 

Analysis of variance of the overall expertise scores (PKST1 & PKST2) of students of different diploma 
type, reveals non-significant differences in expertise level of the three groups. Table 4 shows the mean 
scores of the three groups of students and the analysis of variance statistics. Diploma type ES refers to 
"Economy Students", LS to "Law Students" and OS to "Other diploma type Students" 4 . 

Table 4 : Differences in expertise between ES, LS and OS. 





It) 

ES LS OS 


F 


PF 


MCA 

ES LS OS 


PKST1 
PKST2 


40.17 16.00 24.25 
27.00 10.60 18.50 


2.74 
2.85 


.104 
.097 


12.30 -11.87 -3.62 
7,73 - 8.67 - .77 



But the results of the multiple classification analysis (MCA), as shown in table 4, reveals consistent trends. 
The mean deviation of the mean of ES is always positive and always negative for LS and OS. These 
differences in expertise between ES, LS and OS are in agreement with our earlier findings (Dochy, Valcke 
& Wagemans, 1991b). The fact that the differences are not statistically significant might be due to the 



' ~ p < .001 

' Thit type of students comprises business studies, students following a very varied set of courses and students which final diploma 
type is - a this moment - not predefined. 
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small number of students involved in this study. Since, in the present study, a lot of attention has been paid 
to the optimization of the research instruments - to be more sensitive to measure differences in components 
of expertise - it is interesting to check whether the present composition of expertise components is different 
in economics, law and other students. 

4.2.2 ES and LS possess a different composition of expertise components. 

Based on the former analysis and the results of our earlier study, we expect that there are specific 
differences in the composition of expertise components in ES and LS. As explained earlier, due to the 
specific research population in distance learning, we entered a third category of other diploma type students 
(OS). 

Table 5 : Differences in expertise components between LS, ES and OS 





m 

ES LS OS 


F 


PF 


MCA 

ES LS OS 


SO KST 
OR KST 
MA KST 


11 9.4 10.25 
16 1.2 8.25 
13.17 5.4 5.75 


.19 
3.79 
2.30 


.826 
.053 
.143 


.73 - .87 - .02 
7.00 - 7.80 - .75 
4.57 - 3.20 - 2.85 



The results in table 5 reveal there are no statistically significant differences in the expertise components 
between the students of different diploma types, multiple classification analysis does confirm the trends 
found in the former part of this text. At each level, the ES obtain higher test-scores than the other students. 
Again we repeat that the non-significance of the trends detected might be due to the small number of 
students in the experimental group. The differences between the student types are remarkable ( and nearly 
significant at the 5 % level) for the optimal requisite expertise test (OR KST). 

The latter finding could be of interest when looking at the potential impact of expertise on learning. Has, 
e.g. expertise as defined by the optimal requisite test, an impact on learning subject-oriented knowledge in 
relation to economics ? 

4.3 The impact of expertise on knowledge acquisition. 
4.3.1 Expertise influences the posttest scores. 

In order to measure the impact of expertise on learning new economics knowledge, a subject-oriented 
posttest 5 was administered to all students after an experimental treatment. During this treatment, all 
students received a specific study task. 

To detect the impact of expertise on the knowledge acquisition regression analysis has been used to define 
the extent to which the prior knowledge scores help to explain the variance in the results for the posttest. 

Table 6 : Regression analysis of general expertise scores 





R2 


X explained 


PKST1 


.368 


37% 


PKST2 


.419 


42X 



The results in table 6 indicate that expertise - as measured by the PKST1 or the PKST2 - helps to explain 
37% to 42% of the variance of the posttest results. This impact is statistically significant. Moreover, 
compared to the results obtained in our earlier study (Dochy, Valcke & Wagemans, 1991b), we obtain a 
much higher percentage of explained variance. The detected impact of expertise is comparable to the results 

1 This posttest was a parallel version of the SO KST. Both tests reflect the subject-content dealt with during the study task. 
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found iz literature (Schmidt, 1987). This result also indicates that the optimization of the research 
instruments has an impact on their sensitivity to measure differences in expertise. 

As a consequence, it might be interesting to look at the complex of expertise components to determine what 
type of expertise the high percentage of explained variance stands for. 

4.3.2 Specific expertise components influence the acquisition of economics knowledge. 

If we enter all the testscores in the regression equation, we get the following picture, indicating the % each 
separate expertise test helps to explain the variance in the posttest scores : 

Table 7 : Regression analyais of expertise component scores 





R2 


X explained 


SO KST 


.198 


20X 


OR KST 


.353 


35% 


MA KST 


.218 


22% 



Already at this level we can see that the optimal requisite test (OR KST) explains a very large proportion of 
the variance in the posttest scores. But a further analysis is needed to determine exactly what the specific 
contribution of each expertise component is. In order to do this, a stepwise regression analysis has been 
calculated 6 . The results show that only the optimal requisite expertise test is restrained and entered in the 
regrejsion equation. The test explains 35% of the variance in the posttest results. The scores for the 
mathematics expertise test and the subject-oriented expertise test do not add any relevant (PIN = .05) and 
significant explanatory power. 

Since the instruments, used in this research project, seem to be more sensitive to differences in expertise, a 
further analysis of the general expertise scores might be interesting. Entering the two general measures of 
expertise (PKST1 & PKST2) in the regression equation might give us more support for findings concerning 
the role of specific expertise components. 

The results in table 7 confirm the predominant impact of optimal requisite expertise in studying economics. 
But it is interesting to note that the second general measure of expertise, comprising the optimal requisite 
test scores and the test scores for the subject-oriented expertise test, helps us to explain up to 42% of the 
variance in the posttest scores. This rather high figure helps to get -a- fuller picture of the components of 
expertise in studying economics. 

4.3.3 Students with better expertise obtain higher scores for the posttest scores. 

To check this hypothesis, the results of two sub-groups of students have been used. The scores of 50% of 
the students with the highest scores (H) and the scores of 50% of the students with the lowest scores (L) for 
expertise in general (PKST1 & PKST2) and for each of the specific expertise tests have been used in the 
analysis 7 . We checked whether students with high or low scores for the specific expertise tests, do also 
obtain significantly different results for the posttest 8 . 



* Taking into account the high intercorrelation between the optimal requisite test and the mathematics test, it ia expected that not all 
expertise Korea will be entered in the regreaaion equation. 

' The letter L and H in the table refer to the group with Low results and the group with High scores for each specific expertise test. 
Because of the limited number of students in the experimental population, we could not use the 25% -measure to aplit up low and high 
groupa aa in our earlier research project. The number of students in each aub-group might have been even more restricted. 

' The mean scores of the high and low group for the specific expertise testa are aignificantly different (p < .0005). 
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Table 8 : The impact of low and high scores for expertise tests on the posttest scores 







m 






HCA 




L 


H 


F 


PF 


L H 


PKST1 


10.25 


16.57 


4.907 


.045 


-2.95 3.37 


PKST2 


10.86 


15.25 


1.983 


.183 


-2.34 2.05 


SO KST 


11.57 


15.63 


.888 


.363 


-1.63 1.43 


OR KST 


10.25 


16.57 


4.907 


.045 


-2.95 3.37 


MA KST 


12.14 


14.13 


.360 


.559 


-1.06 .93 



Taking into account the restricted number of students in the experimental group, we do not expect to obtain 
significant differences between the Low and High expertise groups of students. Nevertheless analysis of the 
trends might be interesting. The results in table 8 confirm our expectations. The differences are not 
significant but the trends are consistent : Low expertise students obtain lower results for the posttest than 
high expertise students. 

4.4 Student estimation of their expertise level does not reflect their real expertise level. 

As explained in the theoretical part of this text, earlier research revealed that students are generally unable 
to estimate their own expertise level. In this research project, students could prior to the administration of 
the expertise tests, indicate on a 4-point-scale their degree of estimated expertise (1 =low 4=high). Table 9 
gives the results of the analysis of variance (general expertise level x estimated expertise level). 



Table 9 : The interrelation between estimated and real expertise 





Estimated exp. level 
1 2 3 4 


F 


PF 


HCA 

12 3 4 


PKST1 
PKST2 


41.7 13.0 26.8 26.0 
27.0 9.0 18.6 21.0 


.19 
1.05 


.826 
.416 


13.6 -15.15 - 1.35 - 2.15 
7.8 -10.15 - .55 7.85 



Table 9 shows that the estimated differences in expertise by self-assessment are not reflected in the levels of 
expertise as measured by the tests. Striking is the under-estimation of their real expertise level by the lowest 
self-assessment group. This group (self-assessment expertise level = 1) shows mean total scores on the tests 
which are higher than those for other level groups (see figure 2 and 3). 
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PKST1 



scores mean-28 




real level 



2 3 4 

expertise level 

- ' — selfassessment level 



Figure 2: Difference! between lelf-asaewnent level* and real expertiie leveli (PKST1). 
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scores mean-19 




2 3 
expertise level 



real level 



selfassess.nent level 



Figure 3: Difference! between sclf-aueument leveli and real expertiie leveli (PKST2). 
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5^ CONCLUSIONS. 

The results of this study concerning the quality and impact of expertise in economics has revealed some 
relevant results with implications for the content and organisation of education in the field of economics. 

The first conclusion is that the variable "student type" (ES or LS) has been helpful to detect differences in 
expertise within a population of students. Moreover, these differences could be extrapolated in terms of 
specific components of expertise. Striking were the differences in optimal requisite knowledge and 
mathematics expertise between both sub-populations. This fact strongly contests the multi-functional nature 
of the course "Economics and Money" and implies structural, organisational of educational adaptations of 
this course towards these differences. 

The second important conclusion of this research is that it could be stated that the level of expertise predicts 
- to a certain degree - future learning results of students. It should be noted that the second general measure 
of expertise (PKST2), comprising the optimal requisite test scores and the test scores for the subject- 
oriented expertise test, explains up to 42% of the variance in the posttest scores. Further analysis revealed 
optimal requisite knowledge is an important component of student expertise. 

Further, it was shown that the estimation of expertise level through self-assessment by the students is not 
very reliable. The estimations do not reflect the levels of expertise as measured by the objective tests. 
Concerning the context-transfer approach, the present study confirms the results of our former study 
(Dochy, Valcke & Wagemans, 1991b). 

The results of this study can be helpful to indicate directions for further research, notwithstanding the 
research population is difficult to reach and the experimental group was relatively small. It looks for 
instance promising to analyze in more detail the complex of components in expertise. In this research 
project, expertise components were defined along a "content dimension". In the near future, other 
dimensions, such as psychological and epistemological dimensions, can be put forward to analyze expertise. 
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